FIG. 1 depicts a portion of a pixel grid 100 of a typical CCD or CMOS image sensor. The pixel grid 100 is in raw Bayer format in which each pixel is represented by a discrete color and in which rows contain either red and green pixels or blue and green pixels. Several 13×13 kernel areas are depicted overlaying the pixel grid 100. Kernels (sometimes referred to as masks, templates, or windows) are often used in image processing to perform neighborhood operations. In one example, each kernel operation updates the value of the center pixel of the respective kernel area. Therefore, the kernel that is used for each area is based on the color of the center pixel. The entire CMOS image sensor could contain up to a million or even millions of pixels.
FIG. 2 depicts coefficient locations for exemplary green 200a, red 200b and blue 200c kernel matrices. The coefficients are depicted as a letter (R/G/B) rather than the actual value for the coefficient. The red 200b and blue 200c kernel matrices each have a 7×7 pattern. However, the green kernel matrix 200a has the 7×7 pattern overlaid by a 6×6 pattern. Note that the patterns in FIG. 2 correspond to the pixel grid 100 of the image sensor FIG. 1. To apply the kernel to the image data, each kernel coefficient is multiplied by the value of the corresponding pixel in the region of the pixel grid 100 that is overlaid by the kernel (“kernel region”). Based on these multiplications, the center pixel in each kernel region is updated. For example, the multiplication products are summed together and the sum is used to replace the center pixel value.
Table I shows the number of operations that could be used to perform the kernel computations.
TABLE IKernel ColorKernel Size# of Multipliers# of AddersRed7 × 74948Blue7 × 74948Green7 × 7 and 6 × 68584Total183180
Table I shows that performing the kernel computations requires one multiplier and about one adder for every kernel coefficient, therefore undesirably utilizing considerable gate-level resources.
It may be possible to reduce the gate-level resources by increasing the speed of the clock used to perform the kernel computations. For example, increasing the clock speed by a factor of N could help to reduce the gate-level resources to 1/N by re-using resources during the extra clock cycles. However, realizing such high clock frequencies may be very difficult or impossible, and may also lead to other problems.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.